GENERALIZED SYMMETRIC DIVERGENCE MEASURES AND 

INEQUALITIES 
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Abstract. In this paper, we have studied the following two divergence measures of type 
s: 



V fl (P||Q) = 



and 



m(p\\Q) 



fj s (P||Q) = [ S ( S -l)]- 1 
\j(P\\Q) = t(Pi-qi)ln(%) 



IT s (P[\Q) = [s(s-l)]- 1 



E(p!^ s +p!^|)-2 



i(P\\Q) = h 



n / i-s , l-s \ 
i— 1 x y i—1 x 



T( P|| Q ) = E (a±a)i n (^y 

2 — 1 V 7 



s = 0, 1 

, a? 0,1 
s = 
s = 1 



The first measure generalizes the well known J-divergence due to Jeffreys and 
Kullback and Leibler ^7] . The second measure gives a unified generalization of Jensen- 
Shannon divergence due to Sibson [22] and Burbea and Rao E], and arithmetic- 
geometric mean divergence due to Taneja |27| . These two measures contain in particular 
some well known divergences such as: Hellinger's discrimination, triangular discrimina- 
tion and symmetric chi-square divergence. In this paper we have studied the properties 
of the above two measures and derived some inequalities among them. 



1. Introduction 



Let 



Pi > 0,^^ = 1 L n > 2, 



i=i 



be the set of all complete finite discrete probability distributions. For all P,Q& T n , the 
following measures are well known in the literature on information theory and statistics: 



Hellinger Discrimination (Hellinger [To] ' 



(1) 



1 n 

h(P\\Q) = 1 - B(P\\Q) = ~J2(VPi-V$) 2 

i=i 
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where 

n 

(2) B(P\\Q) = J2Vm~i- 

1=1 

is the well-known Bhattacharyya pQ coefficient. 

• Triangular Discrimination 

(3) A(P\\Q) = 2 [1 - W(P||Q)] = £ ^=^L, 

i=i Pi + 9i 

where 

(4) np\\Q)=£,^> 

i=i p< + gi 

is the well-known harmonic mean divergence. 

• Symmetric Chi-square Divergence (Dragomir et al. [Tl] ) 

(5) ^(phq) = x 2 (png) + x 2 (gi|p) = E fa " qi?{Pl + 

where 

(6) x2 (P|| Q ) = ^(^)! = ^^_i, 

is the well-known x 2 — divergence (Pearson |21j ) 

• J-Divergence (Jeffreys JTH]; Kullback-Leibler [T7| ) 



(7) J(P||g) = M-)- 



Jensen-Shannon Divergence (Sibson Burbea and Rao [21 El) 



(8) /(P||0) = i 



2% 



Pi + ft 



Arithmetic-Geometric Divergence (Taneja [21]) 



(9) T(P\\Q) = Y1 ln 



i=i 



Pi + % \ , / Pi + ft 



2 / \^^/Piq. 



After simplification, we can write 
(10) J(P\\Q)=4[I(P\\Q)+T(P\\Q)}. 
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The measures J(P\\Q), I(P\\Q) and T(P\\Q) can also be written as 



(11) 
(12) 

and 
(13) 
where 
(14) 



J(P\\Q) 

i(p\\Q) 
T(P\\Q) 



K(P\\Q)+K(Q\\P), 



K P 



{ P + Q 



+ K[Q\\ 



P + Q 



K 



P + Q, 



IP 



K 



P + Q 



\\Q 



k(p\\q)=J2p< 1 °z\- 
t=i v<?i 



is the well known Kullback-Leibler relative information. 

The measure © is also known by Jensen difference divergence measure (Burbea and 
Rao IS]). The measure (jTHj) is new in the literature and is studied for the first time 
by Taneja j2Zj and is called arithmetic and geometric mean divergence measure. For 
simplicity, the three measures appearing in (JBJ), © and (fTUJ) we shall call, J S- divergence, J- 
divergence and the AG-divergence respectively. More details on these divergence measures 
can be seen in on line book by Taneja |28j . 

We call the measures given in ©, 0, ©> ©? © an d (fTU|) by symmetric divergence 
measures, since they are symmetric with respect to the probability distributions P and 
Q. While the measures © and (|14j) are not symmetric with respect to probability distri- 
butions. 



2. Generalizations of Symmetric Divergence Measures 

In this section, we shall present new generalizations of the symmetric divergence mea- 
sures given in Section 1. Before that, first we shall present a well known generalization of 
Kullback-Leibler's relative information. 



Relative Information of Type s 

'k s (P\\Q) = [s(s-1T 1 

(15) $ S (P\\Q) = \ k(Q\\P) = £ qi \nhA. 

i=l ^ ' 

K(P\\Q) = £ Pl \n(f 

for all sGi 



i=i 



l-s 
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The measure (fT5|l is due to Cressie and Read [7]. For more studies on this measure 
refer to Taneja [2H] and Taneja and Kumar and reference therein. 

The measure El admits the following particular cases: 

« ^- 1 {P\\Q) = \x\Q\\P)- 

(ii) %{P\\Q) = K{Q\\P)- 

(iii) $ 1/2 (P| \Q) = 4 [1 - B(P\ \Q)) = 4h(P\\Q). 

(iv) UP\\Q) = K(P\\Q). 

(v) <$> 2 {P\\Q) = \ X 2 {P\\Q). 

Here, we observe that $*{P\\Q) = $-i(Q||P) and $i(P\\Q) = $ (Q\\P). 



2.1. J-Divergence of Type s. Replace K(P\\Q) by $ S (P||Q) in the relation (fTT|l. we 

get 



(i6) v.(P||g) = *.(p||g) + $.(g||P) 



J.(P||Q) = [ a (a-l)] 



-i 



E(p^- S +p!"^)-2 



,i=i 



^110 = E (Pi-*) In (|) 



s = 0,1 



The expression (116)1 admits the following particular cases: 

« V- 1 (p\\q) = v 2 (p\\q) = 1 ^(p\\q). 

(uV (P\\Q)=V 1 (P\\Q) = J(P\\Q). 
(iii) V 1/2 (P\\Q) = 8h(P\\Q). 

Remark 1. The expression hlffy is the modified form of the measure already known in 
the literature: 



(17) V]{P\\Q) 



J S (P\\Q) = (s - l)- 1 



i=l 



J(P\\Q) = ±(pi-qi)ln (| 



s ^ l,s > 



s = 1 



For the propertied of the measure JTTp re/er to Burbea and Rao 0111]? Taneja j2Hll2Zll 
eto. For t/ie axiomatic characterization of this measure refer to Rathie and Sheng 
and Taneja . The measures Mb\) considered here differs in constant and it permits in 
considering negative values of the parameter s. 

2.2. Unified AG and JS - Divergence of Type s. Replace K(P\\Q) by $ a (P\\Q) 
in the relation (|13|) interestingly we have a unified generalization of the AG and JS - 
divergence given by 
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;is) w s (p\\Q) 



* s (^IIpW^±% 



2 J \ 2 

i-i 



IT S (P\\Q) = [s(s - 1)Y 



n / i _ B i _ 

^2 ( p * +9 > 



i=i 



WW) 



n / \ n , 

Eftln(^)+Eftln(^j 

4 = 1 x 7 1=1 x 



r(^IIO) = E 

i=l 



)ln 



The measure (118)1 admits the following particular cases: 
(i) W- l (P\\Q) = {A(P\\Q). 

(h) w (p||o) = /(p||g). 

(iii) W 1/2 (P\\Q)=4d(P\\Q). 

(iv) m(p\\Q) = T(p\\Q). 

(v) W 2 (P||Q) = ^(p||q). 

The measure d(P\\Q) given in part (iii) is not studied elsewhere and given by 



(19) 



d(P\\Q) 



i=l 



Pi + v^a / /pi + * 



A relation of the measure ()19|) with Hellinger's discrimination is given in the last 
section. Connections of the measure ()19)) with mean divergence measures can be seen in 
Taneja 



We can also write 
(20) Wi_ a (P||Q) 



$, P 



2 y v 2 

Thus we have two symmetric divergences of type s given by (|T^j) and ([18)1 generalizing 
the six symmetric divergence measures given in Section 1. In this paper our aim is to 
study the symmetric divergences of type s and to find inequalities among them. These 
studies we shall do by making use of the properties of Csiszar's f-divergence. 

3. Csiszar's /-Divergence and Its Properties 



Given a function / : (0, oo) 
is given by 

(21) 



— > M, the f-divergence measure introduced by Csiszar's |S] 

i=i 



for all P, Q e r n . 

The following theorem is well known in the literature [3 Ej • 
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Theorem 1. If the function f is convex and normalized, i.e., f(l) = 0, then the f- 
divergence, Cf(P\\Q) is nonnegative and convex in the pair of probability distribution 
(P,Q)eV n xV n . 

The theorem given below give bounds on the measure (|2T|) . 

Theorem 2. Let f : R + — >• R be differentiable convex and normalized i.e., /(I) = 0. If 
P,Q G T n , are such that 0<r^ — ^i?< oo, Vz G {1, 2, n}, for some r and R with 
0<r^l^i?< oo ; r ^ R, then we have 

(22) *C C f {P\\Q) ^ E Cf (P\\Q) < A Cf (r,R), 
and 

(23) ^ C f (P\\Q) *C B Cf (r,R) <: A Cf (r,R), 
where 



(24) E Cf (P\\Q) = J2& 



(25) ^>,fl) = ±(i2-r)(/'(i2)-/'(r)) 



and 



(26) %M)= MMtMW. 

J R — r 

The proof is based on the following lemma due to Dragomir jH]. 

Lemma 1. Let f : I C R+ -> 1 6e a differentiable convex function on the interval I, 

o o n o 

%i G j (i is the interior of I), \i ^ (i = 1,2, ...,n) X] A « = 1- If rn, M E I and 

i=i 

m ^ Xj ^ M ; Vz = 1,2, ...,n, t/ien we /icwe i/ie inequalities: 



(27) £>/(*<)-/ 



71 

i=l \i=l 
n / n 



^ ^ XiXif'(xi) - I ^ AiSj J I ^ \f\xi 

i=l \i=l / \i=l 

<i(M-m) (f'(M) — f{m)) . 
As a consequence of above theorem we have the following corollary. 
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Corollary 1. For all a,b,v,uj G (0, oo), the following inequalities hold: 



v + uj \ v + uj 

vaf'(a) + ubf'ib) fva + ujb\ fvf'{a) + ujf{b) 



V + UJ \ V + UJ ) \ V + UJ 



<\(b-a) (f'(b)-f(a)). 

Proof. It follows from Lemma^ by taking Ai = -^h;, A2 = ^!hj, A3 = ... = X n = 0, X\ = a, 
x 2 = b, x 2 = ... = x n = 0. □ 

Proof, of the Theorem^ For all P, Q G T n , take x = & in (|2*7)) . Aj = and sum over all 



i = l,2,...,nwe get the inequalities 

Again, take v = R — x, uj = x — r, a = r and b = R in (f2l"j). we get 



(29) Q ^(R-mrn(.-r)UR)_ m 

From the first part of the inequalities ("2"")) . we get 

(R-x)f(r) + (x-r)f(R) 



(30) /(x) < 



i?-r 



For all P, Q G T n , take x = 21 in ("2""") and ([3*0)1. multiply by g« and sum over all 
i — 1,2, n, we get 

(31) 0< J B C/ (r, J R)-C / (P||g) 

(i2-l)(l-r) (/'(«)- /'(r)) 



i2-r 
and 

(32) O^C^PHQJ^Sc^r,^), 
respectively. 

The expression (132)1 completes the i./i.s. of the inequalities (j23j). In order to prove r.h.s. 
of the inequalities ([""*")). let us take x = 1 in (|2*""") and use the fact that /(l) = 0, we get 

(33) O^fMr. W^'-^-^ W). 

J R — r 1 

From (JH3J), we conclude the r./t.s. of the inequalities ("2*3*)) . □ 

Remark 2. VFe observe that the inequalities h22\) and h2ty are the improvement over 
Dragomir's |"J [3101 work. From the inequalities h23\) and hS^I we observe that there is 
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better bound for Bc f {r, R) instead of Ac f (r, R). From the r.h.s. of the inequalities \3'3\) . 
we conclude the following inequality among r and R: 



(34) 



(R-l)(l-r)^-(R-r) 2 . 



Theorem 3. (Dragomir et al. Pud)- («) Let P Q e r « be such that < r < - < 
R < oo, Vz G {1,2, ...,n} ; /or some r and R with 0<r^l^P< oo, r ^ R. Let 
f : [0, oo) —>■ ~R be a normalized mapping, i.e., /(l) = such that f is locally absolutely 
continuous on [r, R] and there exists a, [3 satisfying 



(35) 
Then 

(36) 

and 
(37) 



a <: f"(x) < Vx G (r,P). 



C7 / (P||Q)--J5 0/ (P||g) 



^-(/3-«) x 2 (p||g) 



C>(P||Q) - E* Cf (P\\Q) ^ -(/3 - a)x 2 (P||Q), 
where Ec f (P\\Q) is as given by x 2 (P||<5) os given &?/ (0) and 

(38) E* Gf (P\\Q) = 2E Cf (^±Q\\Q^ = £ (ft - ft)/' (^) 



(ii) Additionally, if f : [r, P] 
[ r , P] 7 ^en 



(39) 

and 
(40) 
where 

(41) 

and 
(42) 



Q(P||Q)-^. (P||Q) 



wzi/i /'" absolutely continuous on [r, P] and /'" G 
1 



^^llrlLixmiQ) 



c>(piiq) - p^(piiq) < - iiriL mwiq), 

(P||Q) = E^^ 



IXI 



II /" 



i=l 



ess sup | f"'\ 

x£[r,R] 



Theorem 4. (Dragomir et al. P3])- Suppose f : [r, P] — > R zs differentiate and f is 



R 



of bounded variation, i.e., V(f') = f R \f"(t)\dt < oo. Let the constants r, P satisfy the 

r 

conditions: 

(i) < r < 1 < P < oo; 

(ii) < r ^ | < P < oo ; for i = 1, 2, n. 
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Then 
(43) 

and 
(44) 
where 
(45) 



C f (P\\Q) - -E Cf (P\\Q) 



R 



<V(f')V(P\\Q) 



1 R. 



C f {P\\Q) - E* Cf (P\\Q) < -V(f')V(P\\Q) 



V(P\\Q) = J2\P*-K 
i=i 



Remark 3. (i) If the third order derivative of f exists and let us suppose that it is either 
positive or negative. Then the function f" is either monotonically increasing or decreasing. 
In view of this we can write 



(46) 
where 

(47) 



P-a= k(f) [f"(R) - f"(r)) 



if f" "is monotonically decresing 
f" i s monotonically increasing 




(ii) Let the function f(x) considered in the Theorem 3.4 be convex in (0, oo), then 
f"(x) ^ 0. This gives 

(48) V(f)= [ R \f"(t)\dt 



f"(t)dt = f'(R) - f(r) 
-A c Ar,R). 



R-r'^ r 

Under these considerations, the bounds and \44\) can be re-written as 



(49) 

and 
(50) 



C f (P\\Q) 



E Cf (P\\Q) 



4 



R-r 



A Cf (r,R)V(P\\Q) 



C f (P\\Q) - E* Cf (P\\Q) 



R-r 



A Cf (r,R)V{P\\Q). 



Based on above remarks we can restate and combine the Theorems El and 0] 

Theorem 5. Let P,Q G T n be such that 0<r^ — ^R< oo, Vi G {1, 2, n}, for some 
r and R with < r < 1 < R < oo. Let f : M. + —>■ M. be differentiable convex, normalized, 
of bounded variation, and second derivative is monotonic with f" absolutely continuous 
on [r,R] and f" G L^r.R], then 
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(51) 



Cf {P\\Q)- l -E Cf {P\\Q) 
^nrfn {!*(/) [f"(R)-f"(r)) X 2 (P\\Q), 

^\\f"\L\x\ 3 (P\\Q), lf(R)-f(r)]V(P\\Q) 



and 

(52) \c f {P\\Q)-E* Cf {P\\Q) 

^vnn\h{f) [f"(R)-f"(r)] X 2 (P\\Q), 

Y A Wf'Woo \x\\p\\Q\ \ [f'(R) ~ f'(r)] V(P\\Q) 

where k(f) is as given by Iffi) . 

Remark 4. The measures (G|), J^i| ) and \45\ j are the particular cases of Vajda 
\x\ m — divergence given by 

n I I m 

(53) \x\ m (P\\Q) = Yl m-i > 

T/ie above measure \5'^l [I] ^T] satisfies the following properties: 

(54) |XP(P||Q) < (1 1} [(1 - r)- 1 + (i? - l)— 1 ] 



sc ( — ^— ) , m ^ 1 

and 

(55) ( ttt ) v ( p HG) < \x\ m (P\\Q) < ( ^zr I v(p||g) ' m ^ L 



Tafce m = 2, 3 and 1 ; m J5^| ) ; we get 



(56) x 2 (P||QKOR-i)(i-rK^-^. 



(57) ixi 3 (p| ig) < \ {R ^ r r) K 1 - + (* - !) 2 ] < - 

and 

(58) ^ll ^y^f' <^-r). 
resj>ec£zi>e/?/. 
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In view of the last inequalities given in |57| ) and 155)) . the bounds given in 157)) 

and hBtyl can be written in terms of r, R as 



(59) 



C f (P\\Q) - -E Cf (P\\Q) 
£ (*Zl£ min \h(f) {f"(R) - f(r)) , 



4 



ii r 



I oo ' 



and 
(60) 



(61) 



R-r 
24 



C f (P\\Q) - E* Cf (P\\Q) 
< { -^^min\h(f) [f'(R)-f"(r)] 



2[f{R)-f{r)] 
R-r 



R-r 
48 



ii r 



rm - f\r) 

R-r 



respectively. 

We observe that the bounds and I160\) are based on the first, second and third order 
derivatives of the generating function. 

Theorem 6. Let fi, f 2 '■ I C M+ — > K two generating mappings are normalized, i.e., 
/i(l) = ^2(1) = and satisfy the assumptions: 

(i) f\ and f 2 are twee differentiable on (r,R); 

(ii) there exists the real constants m, M such that m < M and 



(62) 

then we have 
(63) 



m < 



< M, >0, VxG (r,P), 



m C /2 (P||g) < C7 A (P||Q) < M C A (P||Q) 
Proof. Let us consider two functions 

(64) r] m (x) = fx{x) - m f 2 (x), 
and 

(65) r) M {x) = M f 2 {x)-h{x), 

where m and M are as given by (J62|) 

Since /i(l) = fi{^-) = 0, then r] m (l) = t?m(1) = 0. Also, the functions fi(x) and f 2 (x) 
are twice differentiable. Then in view of ()62jl . we have 

7i(^) 



(66) 



£W = /|'(x)-m/^) = /^) 



m > 0, 
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and 
(67) 



rf M {x) = M fl{x) - f{{x) = fl{x) M 



> 0, 



for all x G (r, R) . 

In view of flfiTlj) and (J57j) . we can say that the functions r) m (x) and t]m{x) are convex on 
(r,i2). 

According to Theorem Q we have 

(68) C^(P||g) = C h „ mh {P\\Q) = C h {P\\Q)-mC h {P\\Q) > 0, 
and 

(69) C VU (P\\Q) = C Mh _ h {P\\Q) = M C h {P\\Q) - C h {P\\Q) > 0. 

Combining © and f!J) we get (jHSJ- □ 

For futher properties of the measure (|21|) based on the conditions of Theorem |H1 refer 
to Taneja 



Remark 5. (i) From now onwards, unless otherwise specified, it is understood that, if 
there are r,R, then 0<r^^^R< oc, Vz G {1, 2, n}, with 0<r<l<R< oo, 

P = (Pi,P2, -;Pn) e T n and Q = (gi,g 2 , ••••,?«) e T n . 

(zz) In some particular cases studied below, we shall use the p-logarithmic power mean 
[21] given by 

p^-1,0 



(70) 



f,p+i_ a p+i 



L p (a, 6) = < 



( p+ i)(b_ a ) 



b—a 



In 6— In a ' 



P = -1 , 



p = 



/or slipeR, 6. In particular, we shall use the following notation 



(71) 



Ll(a,b)={ 



(p+l)(6-a)> ^ 7= 1 > u 
In b— In a 



> h-a ' P- 1 



U, p = o 

for all p 6l, 6. 

4. Bounds on Generalized Divergence Measures 

In this section we shall show that the generalized measures given in Section 2 are the 
particular cases of the Csiszar's f-divergence. Also, we shall give bounds on these measures 
similar to Theorems The applications of Theorem |H1 are given in Section 5. 
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4.1. Bounds on J-Divergence of Type s. Initially we shall give two important 
properties of J-divergence of type s. 

Property 1. The measure V S (P\\Q) is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n for all s G (— oo, oo). 



Proof. For all x > and s G (—00,00), let us consider in 

/ [ S (s ~ IT' [X s + X 1 - - (1 + x)] , 8^0,1 

(72) <f> a (x) = 1 

I (x — 1) lax, s = U, 1 

then we have C f (P\\Q) = V s (P\\Q), where V s (P\\Q) is given by ffToTl. 
Moreover, 



(73) <t>' s {x) 
and 



[s(s - 1)] _1 [six"- 1 + x~ s ) + x~ s - 1] , s ^ 0, 1 
1 — x^ 1 + lux, s — 0, 1 



(74) = X s " 2 + x"^ 1 

Thus we have 4"" s (x) > for all x > 0, and hence, </> s (x) is convex for all x > 0. Also, 
we have </>.,( 1) = 0. In view of this we can say that J-divergence of type s is nonnegative 
and convex in the pair of probability distributions (P, Q) G T n x T n . □ 

Property 2. T/ie measure V S {P\\Q) is monotonically increasing in s for all s ^ \ and 

decreasing in s ^ |. 

In order to prove the above property, we shall make use the following lemma. 

Lemma 2. Let f : I C M + — ► IR 6e a differentiable function and suppose that /(l) = 
f (1) = 0, toen 



(75) /(x 




/ is convex 
if f is concave 

Proof. It is well known that if the function / is convex, then we have the inequality 
(76) f{x){y - x) < f{y) - f(x) ^ f'{y)(y - x), 

for all x, y G R+. The above inequality is reversed if / is concave. Take x = 1 in the 
inequality (f7T)|) and use the fact that f(l) = f'(l) = we get the required result. □ 

Proof, of the Property^ Let use consider the first order derivative of the function <j) s (x) 
given in (|T2|) with respect to s, we get 



(77) k s (x) = A (cf> s (x)) 

as 



[s(s - l)]~ 2 [s(s - l)(x s - x 1 - 3 ) lnx 
+ (1 - 2s)(x s + x 1 - 8 - (x + 1)] , s ± 0, 1. 
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Now, calculating the first and second order derivative of the function k s (x) with respect 
to x, we get 

+s(s - 1) (sx 3 - 1 + (s- l)x~ s ) \nx] , s ^ 0, 1 

and 

k" s ( x ) = (x s ~ 2 -x-'-^hix. 
For all x > 0, we can easily check that 

'>0, 0§ 



< 0, s < ~ 



(78) 

Since k s (l) = k' s (l) = 0, then using Lemma 121 along with (J78j) . we have 
(79) k s {x) 



>0, o| 



Thus from ()79p. we conclude that the function a (a;) is monotonically increasing in s 
for all s ^ | and monotonically decreasing in s for all s ^ | . This completes the proof of 
the property □ 

By taking s = |, 1 and 2, and applying the Property El one gets 

(80) h(P\\Q)<:h(P\\Q)^^(P\\Q). 

Theorem 7. The following bounds hold: 

(81) V S (P||Q) ^ E Va (P\\Q) ^ A Va (r,R), 



:S2) 



V 8 (P\\Q) < B Vs (r,R) < A Va (r,R), 



a) 



V s (P||g)--^ s (P||Q) 



^nnn<|^ Vs (r,it!)x 2 (P||g), ^ IICIL 



and 

(84) |V.(P||Q)-i$,(P||Q)| 

^min|^ Vs (r,P) X 2 (P||g), 1 ||CIL \x\" (P\ \Q), \f {<i>')V {P\\Q) 
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where 



E (Pi ~ Qi 



(85) E Vs (P\\Q) = { 



i=i 



s-l 



[ s _i)-if 2i) - a -Ma 



j(p||g) + x 2 (g||p), 



s = 1 



E (P. - Qi 



[86) EUP\\Q) = { 



i=i 



(5-1)- 



1 / Pi+g» 



2?i 



— S" 



1 / Pj+'Ji 



2(/, 



A(P\\Q) + 2J(?±Q\\Q) 



s = 0,1 



37) 



A Vs (r, fl) = - r) 2 {L-ftr, i?) + LZ s s Z\{r, R) } 



[s(s-l)]- 1 



[88) B Vs (r,R) 



(l-r)(R s +R 1 - s )+(R-l)(r s +r 1 - s ) 



(R-r) 



- 2 



(l-r)(R-l)Lz{{r,R), 



8^0,1 



s = 0,1 



(89) 5 Vs (r, R) = (R- r) [(2 - s)L s s Z 3 3 (r, R) + (1 + s)LZ^(r, i?)] , -1 < s < 2 



(90) 
and 

(91) 



110: 



s II oo 



(2 - s)r s ~ 3 + (s + l)r 



-5-2 



1 ^ s € 2, 



r R — r 



Proof. By making some calculations and applying the Theorem El we get the inequalities 
(18 1|) and (|82j) . Let us prove now the inequalities (|83|) and (|84j) . The third order derivative 
of the function <p s {x) is given by 

(92) <t>"J{x) = -[(2- s)x s ~ 3 + (s + l)x- s - 2 ] , x G (0, oo). 
This gives 

(93) ^'/(x) 0, V - 1 < s < 2. 

From (J$H|) . we can say that the function </>"(x) is monotonically decreasing in x G (0, oo), 
and hence, for all x G [r, i?], we have 

(94) 5 Vs (r,R) = <j ) "(r)- ( j ) "(R) 

= (R-r) [(2 - s)L s s Zl(r, R) + (1 + ^Ll^r, R)} , -1 < * < 2. 
From (|92p. we have 



= (2 - s)x s - 3 + (s + l)x 



,-s-2 



-1 < s < 2. 
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This gives 

(95) \<t>'"(x)\' = -[(s- 2)(s - 3)x s ~ 4 + (s + l)(s + 2)oT 3 - s ] 

< 0, -1 < s ^ 2. 

In view of (J95|) . we can say that the function |0' s "(x)| is monotonically decreasing in 
x G (0, oo) for — 1 ^ s ^ 2, and hence, for all x G [r, R], we have 

(96) IIC'IL = \tf(x)\ = (2 - S )r^ 3 + ( S + l)r— 2 , 

xe[r,R] 



-1 < s ^ 2. 



By applying Theorem El along with the expressions (J9"2j) and (}9l)j) for the measure (|T5j) 
we get the first two parts of the inequalities and (|84*}l. The last part of the inequalities 
(JBHJ) and (jHU) are obtained by using (|9~TJl and Theorem □ 

In particular when s = |, 1 and 2, we get the results studies in Taneja (SHI EH]- 

4.2. Bounds on AG and JS — Divergence of Type s. Initially we shall give two 
important properties of AG and JS - divergences of type s. 

Property 3. The measure W S (P\\Q) is nonnegative and convex in the pair of probability 
distributions (P, Q) G T n x T n for all s G (— oo, oo). 

Proof. For all x > and s G (—00,00), let us consider in (|T5|) 



(97) 



[s(s-l)} 



-1 



1 \ (x+l 



^ s {x) = { f lnx- (2±1) In (2±i) 



(^) (s§) 



s = 



s = 1 



then we have C f (P\\Q) = W S (P\\Q), where W S (P\\Q) is as given by Pjl. 
Moreover, 



(98) 

and 
(99) 



s-1 



1 - x- 1 - In a: - 21n(-| T ) 



3^0,1 



s = 



s = 1 



+ 1\ /s + 1 



8-2 



Thus we have ip's{x) > for all a; > 0, and hence, ip s (x) is convex for all x > 0. Also, 
we have ^ s (l) — 0. In view of this we can say that AG and JS - divergences of type s is 
nonnegative and convex in the pair of probability distributions (P, Q) G T n x T n . □ 
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Property 4. The measure W S (P\\Q) is monotonically increasing in s for all s is — 1. 
Proof. Let us consider the first order derivative of (|57jl with respect to s. 



(100) 



m s (x) = C0 s (x)) 



-s(s- lXx^ + l) In 



x + 1 



s^0,l. 



Now, calculating the first and second order derivatives of (|100|) with respect to x, we 



get 



mAX) 



1 - 2s 
s 2 (l -s)< 



[x s + x 1 ' 8 - (x + 1)] + —, -t(x s - x x ~ s ) lnx, s^0,l 



and 
(101) 

respectively. 



m"Ax) 



2x 2 (x + l) 2 V 2 



x + 1 



a; 1 " 5 In 



s(s-l) 

x + 1 
2x 



+ x In 



x + 1 



Since (x — 1)^0 for any x, this give us 



(102) 



In 



x + 1 
2x 



> In 



x + 1 



Now for all < x ^ 1 and for any s ^ — 1, we have x 1 s ^ x 2 . This together with 
(PUD gives 

(103) m"(x) ^ 0, for all < x < 1 and s ^ -1. 

Reorganizing (|101|) . we can write 



m" s {x) 



x 



2x 2 (x + l) 2 V 2 



x + 1 



x 1+s + 1) In 



x + 1 



— lnx 



Again for all x ^ 1 and s ^ — 1, we have x 1+s + 1^2. This gives 



(104) 



x 1+s + 1) In 



x + 1 



> 2 In 



x + 1 



> lnx, 



where we have used the fact that (x + l) 2 ^ 4x for any x. 
In view of (|lU4j) . we have 

(105) rn" s (x) ^ 0, for all x ^ 1 and s ^ -1. 
Combining (|103|) and (|105|) . we have 

(106) m"(x) ^ 0, for all x > and s ^ -1. 

Since m s (l) = m' s (l) = 0, then ()106j) together with Lemma El complete the required 
proof. □ 
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By taking s = — 1, 0, ~, 1 and 2, and applying Property |U one gets 
(107) 

Theorem 8. The following bounds hold: 



\a(P\\Q) <: I(P\\Q) ^Ad(P\\Q) < T(P\\Q) < ±9(P\\Q). 



(108) 



< W a (P\\Q) < E Ws {P\\Q) < A Wa (r,R), 



(109) 



< W,{P\\Q) < B Wa (r,R) <: A m (r,R), 



(110) 



and 



W S (P\\Q) - -E m (P\\Q) 
^min{i5 Ws (r ;J R) X 2 (P||g), ^ K \L\xf (P\\Q) , yW)V{P\\Q) 



(in) |w.(p||q)-e^(p||q)| 

^minj^^^x^PllQ), ^W:\L\x\ 3 (P\\Q), \v^')V{P\\Q) 

where 

(112) £W.(P||Q) 



i=l 



<J(^||P) 



(^) 



s-l 



1 ( Pi+gz 

2p, 



[i[ x 2 (gi|p)-j(p||g)] + j(^HQ) 



(113) 



Pi+Qi 



s-l 



+ 



s-l 



-1 / P»+3q» 
2g, 



2J( P±Q||P^Q ); 



s = 



[|A(p||g)-ij(^±2||g) + 2j(£f2||g), s = i 
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(114) 



A w .(r, R) 



(R 



16 



—L 

rR 



s-1 

1 



r + 1 R 



2rR 



j s-2 
^s-2 



2r ' 2R 
r + 1 R+l\ 1 



2r 



2r 



, rs-2 

+ 2 L -2 



r + 1 R+1 



(115) By„(r,i2) 

- 1)]- 1 (^) m s 

+(R-1) (^±i) (^) s ]-l}. 
^{(1-r) [i21nfl-(l + i2)ln(^)] 



1) [rln 



r — ir 



l)ln(f)]} 



±{(l-rR)Lz\(r+l,R+l)+\n 



(r+l)(fl+l) 
8 



s = 



s = 1 



(116) 



5 Ws (r, R) 



r s 1 + 1 \ (r + 1 



s-2 



R- 3 - 1 + 1\ R+1 



s-2 



, -1 < s ^ 2, 



d") IW"»»-2(^T)(T) Sx 

x [Sr^ 1 + (s + l)r- s - 2 + (2-s)], -1 < s ^ 2, 

and 

(118) 7(^) = -^-^w.(r,fl). 

r K — r 

Proof. By making some calculations and applying Theorem E] we get the inequalities (|1U8|) 
and (jl(J9|) . Now, we shall prove the inequalities (jllUj) and (Jill)) . The third order derivative 
of the function ip s (x) is given by 

< u9 > «'w-2(iW(T)* x 

x [3X- 3 - 1 + (s + l)x- 2 - s + (2 - sj\ , x E (0, oo). 

This gives 

(120) xp'"(x) < 0, -1 < s < 2. 
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From (|12U|) . we can say that the function <p"(x) is monotonically decreasing in x G 
(0, oo), and hence, for all x G [r, R], we have 

(121) 8w 3 (r,R) = i//'(r) -if/'(R) 

' r -s-l _|_ ]_\ / r _|_ 1 



-1 < s ^ 2. 



ir a-1 + l\ /i? + l xs 2 



Again, from (|119p . we have 

< 122 > 1^)1 = ^TiP^)'" 

x [3X- 8 - 1 + (a + l)ar 2 ~ s + (2 - s)] , x G (0, oo), -1 < s < 2. 

This gives 



(123) | 



x 1 s 



2(z + l) 4 

x {x 1 " 5 [I2x 2 + 8(s + l)x + (s + l)(s + 2)] + x 4 (s - 2)(s - 3)} 
< 0, -1 < s < 2. 



In view of ()123j) . we can say that the function is monotonically decreasing in 
x G (0, oo) for — 1 ^ s ^ 2, and hence, for all x G [r, R], we have 

(124) ||CIL= sup |<'(s)| 

xe[r,.R] 



2 r + 



^(^)* [3 ^ 1 + (s + 1) 



r- 2 - s + (2-s) , -1 ^ s ^ 2 



By applying the Theorem 03 along with the expressions (j!21j) and ()124j) for the measure 
(I18|) we get the first two parts of the bounds (JllUj) and (Jlllj) . The last part of the bounds 
(ITTTTjl and (fTTT]) follows in view of ^TTHj) and Theorem El □ 

In particular when s = —1, 0, 1 and 2, we get the results studied in Taneja |3Till3T] . 

5. Relations Among Generalized Relative Divergence Measures 

In this section, we shall apply the Theorem El to obtain inequalities among the measures 
(USD and PI). 

Let us consider 

025) 9(^,)M = ^) = 8(x -,- 1+I ,- 2) (— j ,*e(0,oo) 

where 4>"(x) and 0"(x) are as given by (J99f and (JUJ) respectively. 
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From (|125|) one has 

(126) 9{^ t) (x): 



x(x + l)(x _ * _1 + x'~ 2 ) 2 
x {aT*- 1 [ X - s - 1 ((t - 2)x + (t-s)) + (t + s- l)x + (t + 1)] 
-a;*" 2 [a- 5 " 1 ((t + l) x + (t + s- 1)) + (t - s)x + (t - 2)] } 



1 / X + 1 



[(*"*)( 



r s-t-2 _ 



8x(x + l)(a;-*- 1 + s'- 2 ) 2 
+ (t-2)(aT s -*- 1 -a;*- 2 ) 

+ {t + s- l)(aT* - x*~ s " 3 ) + (t + l)^-*- 1 - x*~ s ~ 2 

+ (t - 2) [x'- 2 (ar s - 2m - 1)] + (t + s - 1) [x*- s - 3 (x s - 2t+3 - 1)] 
+(t + l) [x*- s - 2 (a;^ 2m -l)]}. 

From the above expression we observe that it is difficult to know the nature of the 
expression (J126j) with respect to the parameters s and t. Here below we shall study the 
above expression for some particular cases of the parameters s and t. 



5.1. Inequalities Among W S (P\\Q) and J(P\\Q). Take t = 1 in (|T23J) and (HHJ), we 

get 

(127) ^ W = i| = 8PTo(-j .*e(0,oo) 
and 

(128) ^a x ) = -^^{^) S 2 [2(x- - 1) + ( S - lMx— 2 - 1)] . 

respectively. 

Using the fact that 



(129) x fc 
we can write the expression (|128j) as follow: 
(130) 



^ 1, x ^ 1, > or i ^ 1, £; < 
^ 1, fc>0ori^l, ^<0 



< 0, (x ^ 1, -2 < s < 0) or (a < 1, s > 1) 
^ 0, (ac < 1, -2 < s < 0) or (x ^ 1, s > 1) 



From (|130|) . we conclude that the function g^^ix) is monotonically decreasing (resp. 
increasing) in (1, oo) and increasing (resp. decreasing) in (0, 1) for all —2 ^ s ^ (resp. 
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s ^ 1). This gives 

(!31) M= sup g^^ix) =^ sA) (l) = ~, -2 < s < 0, 

ze(o,oo) o 

and 

(132) m = inf 0(v,„fc)(x) = S^iK 1 ) = o> s ^ L 

xe(o,co) o 

By the application of the inequality (j63j) given in Theorem El with the expressions 
(|131|) and (|132j) . we conclude the following inequality among the measures W S (P||Q) and 
J(P\\Q): 

Uy(P\\Q), -2<s<0 

(133) Ws(P\\Q)\ 

(>lJ(P\\Q), Ol 

In particular the expression (j!33|) lead us to to the inequality 



(134) I(P\\Q)^lj(P\\Q)^T(P\\Q). 



5.2. Inequalities Among W 8 (P\\Q) and h(P\\Q). Take t = 1/2 in (IT231) and (fM . 

we get 

(135) ^,, 1/2 ) (*) = ^ = ^— j ,xe(0,oo) 
and 

(136) 4 s ,, 1/2) (x) = - 32( ^ 1)a (^) x 

x [aT^ 1 (3ar + (2s - 1)) - (2s - l)x - 3] 

respectively. 

Again in view of ()129|) we can write 

{^0, (x ^ 1, -2 < s < 0) or (x ^ 1, s ^ §) 
• 
^ 0, (x s= 1, -2 ^ s 0) or (x ^ 1, s ^ \) 

From ()137|) . we conclude that the function g(^ 3 ^ 1/2 )(x) is monotonically decreasing (resp. 
increasing) in (1, oo) and increasing (resp. decreasing) in (0, 1) for all —2 ^ s ^ (resp. 
s ^ 1). This gives 

(138) M = sup gw.,4> 1/2 )(x) = £(^ 1/2 )(1) = 5, -2 < s < 0, 

xe(o,oo) o 
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and 

( 139 ) m = 5(^,01/2)^) = iWi/^ 1 ) = ^ s > \- 

xe(o,oo) o z 

By the application of the inequality (J53J1 given in Theorem El with the expressions 
(jl38j) and (jl39j) . we conclude the following inequality among the measures W S (P||Q) and 
h{P\\Q): 



(140) Ws(P\\Q) 



^h(P\\Q), -2^s^0 
>h(P\\Q), s>\ 



In particular this gives 

(141) I(P\\Q)^h(P\\Q)^T(P\\Q). 

The expressions ffHUj) . (jl(J7|) . ()134|) and ()141|) together give an inequality among the six 
symmetric divergence measures as follows: 

(142) \HP\\Q) < I(P\\Q) < h(P\\Q) ^ ±J(P\\Q) ^ T{P\\Q) < ^(P\\Q). 
Remark 6. (i) In view of HJW and il(J\) . we have the following inequality: 



(143) ~A(P||Q) < J(P||Q) < h(P\\Q) ^ ±J{P\\Q) < T(P\\Q) < \j{P\\Q). 
(ii) It is well known jTHj that 

(144) ±A{P\\Q)^h(P\\Q)^±A{P\\Q). 
In view of \14^\ ) and \144D > we have the following inequality: 

(145) \HP\\Q) ^ I{P\\Q) < h{P\\Q) ^ \HP\\Q). 



5.3. Inequalities Among W S (P\\Q), V t (P\\Q) and V(P\\Q). For t = 2, we have 

HP\\Q) = l^(P\\Q) 

and 

W 2 (P\\Q) = ^(P\\Q). 
Since the measure W S (P||(5) is monotonically increasing in s for all s ^ —1. This gives 

U±V(P\\Q), -l<s^2 
(146) Ws(P\\Q)l 

[>ls*(P\\Q), 2 



24 



INDER JEET TANEJA 



Also, in view of monotonicity of Vt{P\\Q) with respect to t, we have 



(147) V t (P\\Q) 



>i*{P\\Q), *<s 



5.4. Inequalities Among V t (P||Q) and A(P\\Q). Take s = -1 in (THZ51) and ([M. we 

get 

(148) = ^ = 2(x - f - 1+x \- 2)(x + 1) . * e (0, oo) 

and 
(149) 



z(a: + l) 4 ^"'" 1 + a;*- 2 ) 2 
x [(* - 2)(x~ t - x 1 - 2 ) + (t + l)^-*- 1 - X*" 1 )] 



z(z + l) 4 (x-*~ 1 + a;*" 2 ) 2 
[(t - 2)x*- 2 (x 2(1 -' } - 1) + (t + l)^- 1 ^- 2 * - 1 



respectively. 

Again in view of ()129|) we can write 



(150) 



^ 0, x ^ 1, * ^ or t < -1 
^ 0, x < 1, i > or i < -1 



From (|15Uj) we conclude that the function g^^^ipc) is monotonically decreasing (resp. 
increasing) in (l,oo) and increasing (resp. decreasing) in (0,1) for all t ^ or t ^ — 1. 
This gives 

(151) M= sup gw_ l7 <i, t ){x) =^_!,^)(1) = J, t ^ or t ^ -1. 

xe(o,oo) o 

By the application of the inequality (|63|) given in Theorem |U] with the expression (|151j) . 
we conclude the following inequality among the measures V t (P||Q) and A(P||Q): 

(152) A(P||Q) < ~V t (P\\Q), 
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5.5. Inequalities Among V t (P\\Q) and I(P\\Q). Take s = in (fT25|) and ffT2fi|). we 

get 

(153) £ (Wt) (x) = |M = 4x(x + 1)(x L 1+xt - 2r - e (0, oo) 
and 

(154) = 2x2(x + l)2(a;-*-i + x*"2)2 X 



X 



[(t - l)(aT« - a;'" 2 ) + ^x"*" 1 - x*" 1 



2x 2 (x + l) 2 ^-*- 1 +x*- 2 ) 2 
[(t - l)^*- 2 ^^ 1 -*) - 1) + tx^ix- 2 * - 1 



respectively. 

Again in view of (J129J) we can write 

(155) g'^y 



x) 



< 0, x > 1 
^ 0, x < 1 



for all t G (— oo, oo). 

From (jl55J) we conclude that the function #(^ o ,0 t )(x) is monotonically decreasing in 
(1, oo) and increasing in (0, 1) for all — oo < t < oo. This gives 

(156) M = sup #(^ t )(x) = g^ a ,<t, t ){l) = -, -oo < t < oo. 

xe(o,oo) o 

By the application of the inequality ()63|) with the expression (j!56|) . we conclude the 
following inequality among the measures V S (P\\Q) and I(P\\Q): 

(157) I{P\\Q) < \v t (P\\Q), -oo < t < oo. 



5.6. Inequalities Among V t (P||Q) and T(P\\Q). Take s = 1 in (fT23|) and (JTHJ), we 

get 

(158) g Mt) {x) = HI = 4x2(x + 1 ) (x + i 1+x ,_ 2r ^ G (0, oo) 
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and 

(159) 9' Mt) (x) = 4 x 4 (x + 1) 2 (x -i-i +x t-2 ) 2 X 



X [t(x 2 - f - X 1 - 2 ) + (t + l)(x - ' - -r 



t+1 _ x t-l) 

+ (t - 2)(x~ t - x f ) + (t- l^x-'- 1 - x t+1 
1 



■X 



Ax 4 (x + l) 2 ^- 1 - 1 + x f - 2 ) 2 
x [tx^ix 2 ^ - 1) + (t + ly- 1 ^ 1 ^ - 1) 

+{t - 2)x\x~ 2t - 1) + (t - l)x m (ar 2 ( m > - 1)] . 



respectively. 

Again in view of ()129j) we can write 



(!60) g{^ t) ( 



^ 0, (x > 1, < t < 1), (x < 1, t ^ 2 or t < -1) 
^ 0, (a: < 1, < t < 1), (z > 1, t ^ 2 or t < -1) 



From (|16Uj) we conclude that the function g(ip lt< f, t )(x) is monotonically decreasing (resp. 
increasing) in (1, oo) and increasing (resp. decreasing) in (0,1) for all ^ t ^ l(resp. 
t ^ 2 or t ^ —1). This gives 

(161) m= sup g(^ t )(x) =g(^ t )(l) = ~, ^ t < 1, 

ze(o,oo) o 

and 

(162) M = inf g(^ t ){x) = g^ 1>( /> t )(l) = \, t ^ 2 or i s= -1. 

£6(0,00) <5 

By the application of the inequality (jHSJ) with the expressions (116 lj) and (|162j) . we 
conclude the following inequality among the measures V t (P\\Q) and T(P\\Q): 



(163) T(P||Q) 



<§V t (P||Q), £^2ori^-l 
>§V t (P||Q), ^ t < 1 



5.7. Inequalities Among W S (P||(3) and V S (P||<5)- When t = s, we shall obtain the 
results in two different ways, one by using the Theorem |H1 and another applying Jensen's 
inequality. 

(i) Take t = s in ([T25|) and ifTCEj) . we get 

W z) = |U = __(_j , ie(0iOo) 
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and 
(165) 



1 



x 



lx(x + l)(x 



s-1 



+ x s 



-2^2 



X + 1 



s-2 



x [(s + l)x" 2 (x 1_ 



X 



s-2/ l-3s 



1) + (s-2)x s ~\x 



1) 



+ (2s - l)x~ 6 (x 



respectively. 

Again in view of ()129|) we can write 



(166) 



9{^ 3 ,<j> s )K x ) 



^ 0, (x ^ 1, | ^ s < 1), (a; ^ 1, s ^ 2 or s < -1) 
^ 0, (a: < 1, | ^ s < 1), (x ^ 1, s ^ 2 or s < -1) 



From ()166j) we conclude that the function gNi a ,<t> s ){%) is monotonically increasing (resp. 
decreasing) in (l,oo) and decreasing (resp. increasing) in (0,1) for all ~ ^ s ^ l(resp. 
s ^ 2 or s ^ —1). This gives 



(167) 

and 
(168) 



m = VP ,9{^,<t>s){x) = g&s^i 1 ) = 77 > 75 < 5 ^ i, 
xe(o,oo) o z 



M = sup g^ 3 ,4, 3 )(x) = gy,.,4.)(l) = O 2 or s ^ -1. 
xe(o,oo) o 



By the application of the inequality with the expressions fll67j) and ()168j) . we 
conclude the following inequality among the measures W S (P||(3) and V S (P\\Q): 

UlV s (P\\Q), 02orO-l 
(169) Ws(P\\Q)\ 

Some particular cases of (j!69|) can be seen in (518) or 



(ii) By applying Jensen's inequality we can easily check that 

^(£i±2i) s ; 0<s<l 



Pi + q s 



(170) 



s>lors<0 



for all i = 1, 2, n, where P, Q G T n . 

Multiplying ()170j) by Pj 1_,s , summing over all i — 1, 2, n and simplifying, we get 



(171) 



i=i 



<Ep!" s (^) s , o< s <i 
i=i 

n 

^EpM 2 ^) 8 . s>lors<0 



8=1 
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Similarly, we can write 

'<£, y ;-* ( ^)\ o<.s<-i 



1 / " 
(172) o l + £tf"'rf 



i=l 



i=l 



i=l 



Adding (|171|) and f)172j) and making some adjustments, we get 
(173) \(^pl- s q! + qt S Pt-2 

< E [ Pl V 1 ) (^) s -i, o< s <i 

^ E ( P '""y a ) (^) 5 -l, s>lors<o' 

Since, s(s — 1) < for all < s < 1 and s(s — 1) > for all s > 1 or s < 0. This 
together with (jl74j) proves that 



(174) V S (P||Q) ^ 4W S (P\\Q), -00 < s < 00, 

where for s = and s — 1, the result is obtained by the continuity of the measures with 
respect to the parameter s. 

In view of (I169j) and (|174j) . we can write 

(175) \v.{P\\Q) < w s (p\\Q) < \v.(p\\q), l^s^i. 

Remark 7. (i) For s = 1 in \175j) , we get a part of the inequality \143j) . For s — 

in Ji75| ) ; we get an interesting bound on a new measure given by Mty in terms of 
Hellinger's discrimination: 

(176) h(P\\Q)^d(P\\Q)^h(P\\Q). 

(ii) In particular for s = \ and t = 1 in A 12 '5)) . we can easily show that 

(177) Ad{P\\Q)^ l -J{P\\Q). 

The inequalities \14 6 4) together with \17b\l and \17 r /\j give the following inequalities 
among some particular cases of the measures Mb]) and hlty : 

(178) \hp\\Q) < I{P\\Q) < WIG) < 4d(P||Q) 

<^J(P||g)<r(P||Q)^^*(p||g). 
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